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2.  STATEMENT  OF  OBJECTIVES 

Airport  baggage  screeners  must  quickly  study  a  cluttered  x-ray  image  for  a  large  number  of 
predetermined  items.  If  they  see  an  unidentifiable  item,  they  must  decide  whether  it  requires 
further  scrutiny  This  is  an  extreme  example  of  the  kinds  of  visual  tasks  we  encounter  each  day 
and  perform  effortlessly  as  we  move  our  eyes  to  likely  target  locations  or  interesting  features. 
How  does  our  visual  system  combine  information  across  space  and  determine  which  locations 
are  useful  for  the  task?  In  this  proposal  we  investigate  how  humans  combine  multiple  sources 
of  information  under  conditions  of  information  overload,  time  pressure  and  degradation  in  the 
periphery.  We  develop  a  model  that  computes  uncertainty  over  space  with  respect  to  the  task 
at  hand,  and  predicts  the  sequence  of  regions  an  observer  will  select. 

Aim  1:  Find  multiple  targets  in  clutter 

In  this  aim  we  compare  human  and  model  strategies  when  the  task  is  to  estimate  the  number 
and  locations  of  independent  targets  in  a  noisy  display. 

a.  Search  for  static  targets  while  fixating:  In  this  experiment,  observers  fixate  a 
central  marker  as  they  search  for  multiple  targets.  They  are  asked  to  report  the  number  of 
targets  and  to  rate  how  confident  they  are  that  a  target  was  present  at  a  particular  location.  How 
does  their  criterion  change  as  we  manipulate  the  noise,  probability  of  target  occurrence  or 
penalty  for  missing  a  target?  Does  the  performance  correlate  with  our  model  predictions? 

b.  Determine  the  location  of  targets  with  eye  movements:  We  repeat  the  first 
experiment  but  now  allow  the  observer  to  move  his  or  her  eyes  while  searching  We  study  the 
observers’  selection  decision  by  recording  eye  movements  and  comparing  them  to  our  model. 
Do  observers  select  maximally  uncertain  regions  as  predicted? 

c.  Active  search  for  dynamic  targets:  When  we  allow  eye  movements,  we  must 
integrate  the  new  visual  information  that  is  being  acquired.  This  update  becomes  even  more 
challenging  when  the  display  is  dynamic,  with  targets  moving  in  straight  paths  amidst  dynamic 
noise,  Observers  will  be  asked  to  report  the  number  of  targets  and  to  locate  them.  Are  several 
saccades  planned  initially,  or  do  we  continually  incorporate  new  information?  Does  the  future 
location  of  the  target  influence  our  decisions? 

Aim  2:  Learn  a  novel  shape 

This  task  is  more  challenging  because  local  information  is  no  longer  independent  over  space, 
but  is  constrained  by  the  context  of  the  entire  shape. 

a.  Search  for  targets  along  a  closed  contour:  Observers  are  asked  to  report  the 
number  of  target  shape  segments  on  a  noisy  circular  profile  that  is  equidistant  from  fixation.  We 
parametrically  alter  noise  amplitude  to  reduce  target  visibility  and  compare  performance  to  Aim 
la.  Does  the  lack  of  independent  information  affect  performance? 

b.  Determine  context-dependent  resolution  of  orientation:  Preliminary  experiments 
show  that  the  resolution  of  a  small  part  of  the  contour  is  not  determined  simply  by  eccentricity.  It 
depends  critically  on  both  the  local  smoothness  of  the  surrounding  contour  and  on  the  global 
shape  of  the  figure  We  quantify  this  effect  and  incorporate  it  into  our  model. 

c.  Predict  the  sequence  of  eye  movements  while  learning  a  novel  shape:  We  will 
modify  our  model  to  incorporate  our  findings  about  context-dependent  resolution  and  the  time 
required  to  update  information  from  a  saccade,  and  a  cost  for  making  large  saccades.  We  will 
test  our  model  predictions  by  comparing  them  with  human  eye  movements  during  shape 
learning 


3.  STATUS  OF  EFFORT 

Our  goal  was  to  determine  efficient  ways  to  combine  information  from  multiple  sources.  In  Aim 
1  we  conducted  experiments  to  determine  human  performance  when  faced  with  the  task  of 
finding  multiple  targets  under  noisy  conditions  and  time  pressure.  The  displays  were  were  too 
brief  to  inspect  all  locations  visually  so  human  observers  had  to  choose  carefully  the  locations 
they  inspected.  We  measured  task  performance  as  well  as  eye-scanning  strategies  as 
observers  performed  these  tasks  We  then  compared  human  performance  and  eye  movements 
to  the  predictions  of  the  optimal  model  for  these  tasks — a  model  that  maximizes  the  information 
gained  on  each  eye  fixation.  Our  model  predicts  that  the  most  efficient  strategy  is  to  direct 
fixations  to  regions  of  maximum  uncertainty,  i.e.  to  regions  that  would  maximize  the  information 
gained. 

Predictably,  performance  declined  with  increasing  noise  level.  More  interestingly,  the  locations 
that  humans  inspected  changed  as  a  function  of  signal-to-noise  ratio  (uncertainty).  At  low  noise 
levels,  observers  did  not  move  their  eyes,  as  the  signal  was  visible  without  eye  movements.  At 
higher  noise  levels,  our  data  indicate  that  they  moved  they  eyes  first  to  the  locations  that  could 
not  be  clearly  distinguished  as  signal  or  as  noise  regions.  This  finding  is  consistent  with  our 
model,  which  predicts  that  the  most  efficient  strategy  is  to  direct  fixations  to  regions  of  maximum 
uncertainty,  i.e  to  regions  that  would  maximize  the  information  gained. 

In  Aim  2  we  measured  human  ability  to  learn  a  novel  shape  under  time  pressure:  We 
used  novel  silhouettes  that  were  large  and  presented  in  the  periphery,  So  the  task  required  the 
operator  to  gather  information  from  multiple  parts  of  the  contour.  Furthermore,  time  pressure 
forced  the  observer  to  gather  this  information  efficiently.  For  the  simplified  case  of  the 
silhouettes  that  we  used,  the  stimulus  information  was  simply  the  orientations  along  the  contour 
of  the  shape,  The  most  uncertain  regions  corresponded  to  parts  of  the  stimulus  that  were  distant 
from  fixation,  and  that  had  multiple  orientations  within  a  neighborhood.  We  compared  human 
eye  movements  to  the  predictions  of  three  models  the  information  gain  model  described  above, 
a  random  model  where  fixation  locations  were  selected  randomly,  and  a  saliency  model  where 
fixations  were  made  to  the  distinctive  features  that  were  easily  discriminated  from  their 
surroundings.  Our  results  showed  that  human  eye  movements  in  the  shape  learning  tasks 
preferentially  went  to  locations  about  which  the  observer  was  most  uncertain 
We  also  examined  whether  eye  movements  became  more  efficient  when  costs/penalties  were 
involved  Performance  improved  when  observers  were  trained  with  a  penalty  for  inefficient  eye 
movements,  but  the  magnitude  of  improvement  was  only  about  5%.  A  more  effective  way  of 
improving  performance  is  to  guide  humans  to  make  eye  movements  to  locations  predicted  by 
the  optimal  model  These  results  have  implications  for  using  the  model  both  for  long-term 
training  and  for  on-line  assistance  to  the  human  operator 

4.  ACCOMPLISHMENTS/NEW  FINDINGS 

We  have  three  conclusions  from  our  study 

1 .  Eye  movements  are  task  dependent.  When  observers  try  to  gather  as  much  information 
as  possible  in  a  brief  stimulus  presentation,  they  do  not  look  necessarily  at  the  most 
salient  points,  as  described  by  other  studies, 

2.  Observers  move  their  eyes  to  locations  that  reduce  uncertainty  about  the  stimulus.  Thus, 
they  appear  to  use  efficient  strategies,  although  their  performance  falls  short  of  optimal. 

3.  We  also  show  that  task  performance  improves  considerably  when  humans  are  guided  to 
make  eye  movements  to  locations  predicted  by  the  optimal  model.  Therefore  our  study 
has  implications  of  using  the  model  both  for  long-term  training  and  for  on-line  assistance 
to  the  human  operator. 
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